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© image editing system and method have improved color key editing. 



© An image editor for an image processing system 
has a predefined set of system colors. Color pixel 
color index values for an image to be edited are 
stored in an image buffer (61). A color lookup regis- 
ter bank (47) stores color values in respective regis- 
ters for a predefined set of system colors. Each 
buffer index value corresponds to a lookup register 
having the specified color value. The image is thus 
displayed by indexing the image pixels to the lookup 
register bank for the color signal inputs to the dis- 
play. A mouse (52) is used to designate displayed 



colors that a user desires to modify in a region of 
the displayed image. Pixel position data correspond- 
ing to the designated colors is detected and hue, 
saturation and brightness (HSB) values are deter- 
mined from the stored color values for the des- 
ignated image pixels. A minimum range of values is 
determined for each of the HSB variables. A 
preselected color edit operation is applied to those 
image pixels having colors corresponding to HSB 
values within the respective minimum HSB ranges. 
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The present invention relates to image pro- 
cessing systems and methods and more particu- 
larly to image editors that employ color key editing. 

Various kinds of image processing systems 
may employ color key editing or selection in image 
editing operations. 

Color key selection uses pixel values to deter- 
mine which pixels are to be modified in accordance 
with an editing operation. Editing or colorizing is 
then performed on all of the selected pixels 
(typically, defined by a bit mask) in accordance 
with a conventional editing operation such as TV 
Image or Line Art. The selected pixels are chosen 
by some suitable means. 

Color key editing is often defined as a proce- 
dure in which pixel selection and colorizing are 
combined in a pixel-by-pixel operation. 

One type of image processing system is a 
duplicator system in which a stored electronic im- 
age can be edited and applied to a platen to 
produce a hard copy output. Another system is a 
reprographic system in which a digital image is 
scanned from a hard copy input and then pro- 
cessed through an image editor to generate an 
edited hard copy output from a laser printer or the 
like. In an image edit system, an image editor 
processes stored images in accordance with edit 
operations and stores the edited images electron- 
ically for use in other systems. 

Color editing generally is a process in which 
some means is provided for defining image pixels 
to which color changes are to be applied. 

However, color editing typically has been limit- 
ed to adjusting all of the color lookup table entries 
that correspond to pixels in a well-defined closed 
region of an encoded form of the full color image. 
Editing productivity and editing quality have ac- 
cordingly been adversely affected. Thus, it may be 
difficult to define a region having only the colors 
that are to be changed, and as a result some 
colors may not be changed that should be changed 
or vice versa. 

For example, consider an image of a tree with 
sky showing through the leaves. The sky color may 
be greenish because of erroneous use of tungsten 
indoor film for outdoor picture taking. As a result, 
tiny bits of sky showing through small spaces be- 
tween the leaves are off-color. Correcting the sky 
color of the small tree spaces without washing out 
the spaces is at best very difficult with known prior 
art procedures 

In any case, the known prior art has had no 
capability for adaptive editing where editing opera- 
tion on a pixel depends on the pixel color. 

The following prior patents have been noted: 
1. U.S. Patent No. 4,167,729, "Apparatus For 
Obtaining Multi-Spectral Signatures", dated Sep- 
tember 11, 1979, filed by Dempster S. Christen- 



son, et al. 

2. U.S. Patent No. 4,488,245, "Method And 
Means For Color Detection And Modification", 
dated December 11, 1984, filed by George W. 

s Dalke, et al. 

3. U.S. Patent No. 4,649,423, "Method and Cir- 
cuit Arrangement For Selective Correction Of 
Hues And Colors", dated March 10, 1987, filed 
by Ingo Hoffrichter, et al. 

io 4. U.S. Patent No. 4,727,425, "Pixel Color Modi- 

fication Using Look-Up Tables In Image Re- 
production System", dated February 23, 1988, 
filed by Clive L. Mayne, et al. 

5. U.S. Patent No. 4,731,662, "Image Processing 
15 Method For Processing An Image Signal Dif- 
ferently Depending On The Range Of An Image 
Characteristic Thereof Relative To The Range 
Within Which An Output Device Can Reproduce 
The Image Characteristic", dated March 15, 

20 1988, filed by Yoshiro Udagawa et al. 

6. U.S. Patent No. 4,758,885, "Method of Pro- 
cessing Color Image", filed July 19, 1988, filed 
Takashi Sasaki, et al. 

7. U.S. Patent No. 4,989,080, "Color Correction 
25 Device With A Hue Area Judgment Unit To 

Determine Correction Parameters", filed January 
29, 1991, filed by Takanori Ito. 

8. U.S. Patent No. 4,992,861 , "Color Image Re- 
production Apparatus Having A Digitally Oper- 

30 ated Look-Up Table Constructed by Means of a 
Least Squares Algorithm", dated February 12, 
1991, filed by John R. D'Errico. 

9. U.S. Patent No. 4,992,864, "Image Processor 
Having Density Indicator", dated February 12, 

35 1991, filed by Kazuya Akiyama. 

10. U.S. Patent No. 5,012,333, "Interactive Dy- 
namic Range Adjustment System For Printing 
Digital Images", dated April 30, 1991, filed by 
Hsien-Che Lee, et al. 

^0 It is an object of the present invention to pro- 

vide an improved color image editor, enabling color 
editing to be performed adaptively in dependence 
on the original color value of each selected image 
pixel. 

45 The present invention provides an image editor 

for an image processing system having a predefin- 
ed set of system colors and comprising means for 
storing color pixel data for an image to be edited; 
means for displaying the stored image on a moni- 

50 tor screen ;means for designating displayed colors 
that a user desires to modify in a region of the 
displayed image; means for detecting pixel position 
data corresponding to the designated colors; 
means for determining and storing values for at 

55 least one color variable from the color pixel data for 
the pixel position data; means for determining a 
minimum range of values for the color variable that 
encompass colors designated by the designating 
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means; and means for applying a preselected color 
edit operation to those image pixels having colors 
corresponding to those color variable values within 
the minimum range. 

In an image editor in accordance with the in- 
vention, said determining and storing means deter- 
mines and stores respective values for multiple 
color descriptor variables; said range determining 
means determines a minimum range of values for 
each of the color descriptor variables; and said 
applying means applies the edit operation to image 
pixels having colors corresponding to those color 
descriptor variable values within at least one of the 
minimum ranges. 

In an image editor in accordance with the in- 
vention, said designating means includes means 
for specifying successive image positions having 
the desired color(s); and said position detecting 
means detects pixels in a small image area around 
each specified position. Said determining and stor- 
ing means determines and stores respective values 
for multiple color descriptor variables; said range 
determining means determines a minimum range 
of values for each of the color descriptor variables; 
and said applying means applies the edit operation 
to image pixels having colors corresponding to 
those color descriptor variable values within the 
respective minimum ranges. A mouse may specify 
the successive image positions in a click and drag 
operation. 

Preferably, the color descriptor variables are 
hue, saturation and brightness. In that case, said 
color pixel data storing means includes an image 
pixel buffer and a lookup color register bank; said 
lookup color register bank having a modifiable reg- 
ister for storing basic color component values for 
each color in the set of system colors; said image 
pixel buffer having image pixel registers each of 
which is indexed to a particular lookup color regis- 
ter to display the color required for that pixel; said 
edit applying means includes means for testing 
each of said modifiable look-up table registers to 
determine those stored color values that corre- 
spond to hue, saturation and brightness values 
within the minimum ranges therefor; and said edit 
applying means further includes means for modify- 
ing those modifiable look-up table registers having 
in-range hue, saturation and brightness values in 
accordance with the edit operation. Said designat- 
ing means includes means for specifying succes- 
sive image positions having the desired color(s); 
said position detecting means detects pixels in a 
small image area around each specified position; 
means are provided for determining respective 
average values of hue, saturation and brights (HSB) 
for pixels in each of said small image areas; and 
said range determining means determines a mini- 
mum range of average values for each of said HSB 
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descriptor variables for application to full color edit 
operations. Said determining and storing means 
includes means for converting said color pixel data 
for said pixel position data to hue, saturation and 
5 brightness values. Said applying means applies the 
edit operation to image pixel colors corresponding 
to those color descriptor variable values within all 
of the minimum ranges. 

As an alternative, said designating means in- 
to eludes means for specifying successive image po- 
sitions having he desired color(s); said position 
detecting means detects pixels in a small image 
area around each specified position; means are 
provided for statistically analyzing color descriptor 

75 variable values to provide respective statistical val- 
ues for each descriptor variable for pixels in each 
of said small image areas; and said range deter- 
mining means determines a minimum range of 
statistical values for each of said color descriptor 

20 variables for application to full color edit operations. 
Said statistical analyzing means provides average 
values for each descriptor variable. 

In one form of image editor in accordance with 
the invention, said one color variable is a color 

25 descriptor variable having values defined in a cir- 
cular domain; and said range determining means 
includes: means for sorting said stored values for 
said variable in value order and for determining the 
greatest value distance between adjacent stored 

30 values A and B; means for comparing the A and B 
values to determine whether the selected range is 
or is not a wraparound range; and said applying 
means includes means for determining whether 
each pixel has a color corresponding to a color 

35 descriptor variable in the selected wraparound or 
non-wraparound range. In a specific case, the color 
descriptor variable is hue and said applying means 
determines each pixel hue value to be in range if it 
is greater than the B hue value and less than the A 

40 hue value if the selected range is a non- 
wraparound range; and said applying means deter- 
mines each pixel hue value to be in range if it is 
greater than the B hue value or less than the A hue 
value if the selected range is a wraparound range. 

45 The present invention further provides a range 

tracker for a variable having values in a circular 
domain, said tracker comprising: means for sorting 
stored values of said variable in value order and for 
determining the greatest value distance between 

50 adjacent stored values A and B; means for compar- 
ing the A and B values determine whether the 
selected range is or is not a wraparound range; and 
means for determining whether each value in an- 
other set of values for said variable is in the se- 

55 lected wraparound or non-wraparound range. In a 
specific case, said determining means determines 
a value to be in range if it is greater than the B 
value and less than the A value if the selected 

3 
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range is a non-wraparound range; and said deter- 
mining means determines a value to be in range if 
it is greater than the B value or less than the A 
value if the selected range is a wraparound range. 

The present invention also provides a method 5 
for editing an image in an image processing sys- 
tem having a predefined set of system colors, the 
steps of said method comprising: storing color 
pixel data for an image to be edited; displaying the 
stored image on a monitor screen; designating w 
displayed colors that a user desires to modify in a 
region of the displayed image; detecting pixel posi- 
tion data corresponding to the designated colors; 
determining and storing values for at least one 
color variable from said color pixel data for said 75 
pixel position data; determining a minimum range 
of values for said color variable that encompass 
colors designated by said designating step; and 
applying a preselected color edit operation to those 
color variable values within the minimum range. 20 
Said determining and storing step determines and 
stores respective values for multiple color descrip- 
tor variables; said range determining step deter- 
mines a minimum range of values for each of the 
color descriptor variables; and said applying step 25 
applies the edit operation to image pixels having 
colors corresponding to those color descriptor vari- 
able values within the respective minimum ranges. 
Preferably, the color descriptor variables are hue, 
saturation and brightness. 30 

By way of example only, an embodiment of the 
invention will be described with reference to the 
accompanying drawings, in which: 

FIGURE 1 shows a block diagram for a general- 
ized image processing system in which an im- 35 
age editor in accordance with the present inven- 
tion is employed; 

FIGURE 2 shows a block diagram for a reprog- 
raphic system which is a particular form of the 
image processing system of FIGURE 1; 40 
FIGURE 3 shows another block diagram for a 
network based image processing system em- 
bodying the present invention; 
FIGURE 4 illustrates a more detailed block dia- 
gram for a computer work station employed in 45 
the network based system of FIGURE 3; 
FIGURES 5A and 5B show editing controls and 
selector menus employed on the screen of the 
work station monitor; 

FIGURE 6A shows a flow chart for a main pro- 50 
cedure employed by the color key editing con- 
trol; 

FIGURE 6B shows a flow chart for a pixel value 
conversion procedure used in the procedure of 
FIGURE 6A; 55 
FIGURE 6C illustrates an entry list data structure 
used in operation of the color key editing con- 
trol; 



FIGURE 6D graphically illustrates range tracking 
for hue; 

FIGURE 6E shows a flow chart for a wraparound 
range tracking procedure used in the procedure 
of FIGURE 6A; 

FIGURE 6F illustrates a procedure used in the 
procedure of FIGURE 6A to determine whether 
a hue value lies in a tracked hue wraparound 
range; 

FIGURE 6G shows a flow chart for a procedure 
used in the procedure of FIGURE 6A to track 
saturation or brightness range; 
FIGURE 6H is flow chart for determining wheth- 
er a brightness or saturation value is within a 
tracked range; 

FIGURE 6J shows a flow chart for a procedure 
used in the procedure of FIGURE 6A to cal- 
culate average values of hue, saturation, and 
brightness; 

FIGURE 6K illustrates a procedure for comput- 
ing average hue for the procedure of FIGURE 
6A; 

FIGURE 6L shows a procedure for modifying 
color lookup tables that are used in applying an 
image edit operation; 

FIGURES 7A-7C provide additional illustrations 
related to determining whether a hue value is 
within a tracked hue wraparound range; and 
FIGURES 8A-8D show window display results 
for four different exemplary cases of color key 
editing. 

There is shown in FIGURE 1 a general image 
processing system 10 which is representative of 
the various kinds of systems in which the present 
invention can be embodied. Image processing sys- 
tems may be used for various purposes including 
the generation of images in the computerized pro- 
cess of creating newspapers, magazines, catalogs, 
brochures or other documents. High image quality 
is normally required to meet advertising artwork 
and other graphic needs in these processes. 

The system 10 includes an image data source 
12 that provides image data for an image proces- 
sor 14 having an image editor. Processed image 
data is applied to an output device 16 such as a 
monitor, a laser or other printer, or a duplicating 
machine platen. 

The image data may be organized for high 
quality, high density color displays. For example, in 
the preferred arrangement, the combined basic col- 
ors, i.e., red, green and blue, of each image ele- 
ment or pixel are stored as a total of twenty-four bit 
values. Each basic color can have 2 s or 256 values 
from 0 to 255. Accordingly, the total number of 
possible colors for each pixel in a color image is 
256 3 or greater than 16 million. Effective, fast, 
quality image editing is achieved with computer 
capacity economy by encoding stored image data 
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to a reduced color set, in this case a set of 100 
preselected colors. 

In FIGURE 2, there is shown a specific kind of 
image processing system 10, namely a reprog- 
raphic system 18. The reprographic system 18 
may be a duplicating system which has an image 
editing capability. 

An image scanner 20 detects image data from 
a hard copy original 22. The image data is placed 
in 24-bit storage 23 and then processed through an 
image editor system 24. An encoder 25 produces 
image data based on a reduced color set, and the 
reduced image data is applied to an interactive 
image editor 26. 

The image edit system 26 processes the 
stored image to produce a modified image result- 
ing from editing operations. A color monitor 28 and 
keyboard 29 are interactively operated with the 
image editor 26 in the editing process. Inputs are 
applied from the 24-bit storage 23 and the inter- 
active image editor 26 to a background image 
editor 27. The edited image is applied from the 
background image editor 27 to an output device, in 
this case a laser printer 30 that will produce a 
requested number of copies. 

In FIGURE 3, another kind of image processing 
system 10 is shown, namely a networked system 
32 having a plurality of image editing computer 
workstations 34a and 34b with associated color 
monitors 35a and 35b. A network 36 for the system 
32 additionally has coupled to it an image storage 
apparatus 38, an image server 40 and a print 
server 42, and, if desired, various other sub- 
systems. On request, the image server 40 provides 
various more complex image processing services 
to the work stations. The networked system 32 is 
the type of system in which the embodiment of the 
invention described below is implemented. 

IMAGE EDITING WORK STATION 

An image editing work station 34, correspond- 
ing to the work station 34a or 34b in FIGURE 3. is 
shown in greater block diagram detail in FIGURE 4. 
A color monitor 35 provides a display in accor- 
dance with red, green, and blue color signals gen- 
erated by drive circuit means. The drive circuit 
means preferably comprises a digital-to-analog 
converter 43 on a display card 44. 

As indicated in FIGURE 4, a digital color signal 
for each screen pixel comprises digital red, green 
and blue signals supplied by lookup color registers 
47. Each of the three components of the color 
signal is an 8-bit signal. Successive screen pixel 
signals are converted to analog signals by the 
converter 43 and rastered onto the monitor screen 
35 in the conventional manner. 



A digital computer system 46, such as a Sparc 
Station II preferably operates the display card 44 to 
generate the monitor display. In other applications, 
the functional interface between the digital com- 
5 puter system 46 and the display card 44 may be 
varied from that which exists in this embodiment. 

Programmed operation of the computer system 
46 occurs under the direction of an operating sys- 
tem 48 such as UNIX. Library support and other 

to procedures 50 are included in the computer pro- 
gram system as needed for the image processing 
application and for system operation. 

An operator input control system 52 includes 
means for entering image editing actions and for 

75 otherwise directing the computer system operation. 
Preferably, the operator input control 52 includes a 
mouse (not specifically indicated) which provides 
various kinds of operator editing entries through 
interaction with the display on the color monitor 35. 

20 The preferred operator control also includes a key- 
board for operator entry of alphanumeric editing 
entries. In other applications, other operator control 
devices, such as light pens, pressure sensitive 
pads, etc., can be employed. 

25 Images to be processed by the work station 34 

are suitably stored. Each image is formed by a 
predetermined number of pixels each of which is 
typically and preferably herein defined by three 8- 
bit words respectively for the red, green and blue 

30 color components. 

When a stored image 54 is called by the 
operator for edit processing, the image 54 is first 
processed by an image encoder 56 to encode the 
image with a reduced set of preselected colors for 

35 effective yet economic image editing for reasons 
previously explained. A conventional image algo- 
rithm program, such as Floyd-Steinberg error diffu- 
sion, is employed for the image encoder 56. 

Image editing is preferably facilitated in the 

AO work station 34 by the use of a windowed display 
on the screen of the color monitor 35. A conven- 
tional windows manager control 58, such as Win- 
dowing System, as developed by MIT, is employed 
in the computer program system to control the 

45 basic windowed structure of the monitor display. 

Image encoded editing means is included in 
the work station 34 to enable the image 54 to be 
modified in various ways. The image editing means 
preferably includes the operator input control sys- 

50 tern 52, various computer editor controls 60, and 
an image storage means preferably including a 
lookup color register bank 47 and an image pixel 
buffer 61 on a display card 44. 

As previously indicated, prior art techniques for 

55 editing images have typically employed procedures 
having limited productivity. This embodiment of the 
present invention, as described below, provides 
significantly enhanced productivity through the im- 
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age editing means which is characterized by im- 
proved structure and operation. 

The encoded stored image provided by the 
image encoder 54 is handled by an image display 
window control 62 for screen display in an image 
window 63 (see FIGURE 5A). The size and location 
and screen pixel assignments of the image window 
63 are determined by the windows manager 58. 
The color content of each image pixel in the image 
window 55 is defined by the image pixel buffer 61 
in combination with the lookup color register bank 
47. 

The lookup color register bank 47 contains n 
registers including a group of registers that respec- 
tively define the R, G, and B values of the respec- 
tive colors in the set of colors preselected for 
system use. The color lookup register assigned to 
a particular image pixel generates the output color 
signals that produce the screen pixel correspond- 
ing to the image pixel. 

The image display window control 62 loads the 
encoded image into the image buffer 61. The en- 
coded image includes an index value for each 
image pixel pointing to a color lookup register that 
contains the R, G, and B values corresponding to 
the color for that pixel. As indicated in FIGURE 4, 
the image pixel buffer 61 stores the number of the 
lookup register specified for each image pixel. Im- 
age pixels are thus processed for screen display 
by reference to the designated lookup registers in 
the pixel buffer 61 . 

The editor controls 60 also include a pixel 
selector window control 64 that is operated through 
a selection menu window 59 (FIGURE 5B) and 
provides the operator with options for limiting or 
providing boundaries on the extent to which the 
image stored in the buffer 61 is to be subjected to 
color and other edit actions. In the preferred em- 
bodiment, the optional selectors include: 

a. Brush. 

b. Freehand. 

c. Rectangle. 

d. Polygon. 

e. Color key. 

f. Automatic object selector. 

Once a pixel selector is chosen by the operator, 
color editing of the image is limited to pixels de- 
fined by the chosen selector. 

An editor window control 66 implements var- 
ious editorial operations selected by the operator 
from an editor menu window 63 (FIGURE 5A). In 
the preferred embodiment, editorial operations or 
user procedures that can be selected include: 

a. TV color. 

b. Line art. 

c. Image color. 

d. Hi-light. 

e. Rotate. 
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f. Crop. 

g. Scale. 

h. Filter. 

In many of the user procedures, operator se- 
5 lections in an active box in the editor menu window 
are implemented in real time on the image in the 
image window. In other cases, operator selections 
are displayed in a separate box in the editor menu 
window and then implemented on the image in the 
io image window upon operator command. 

In the various edit operations, color modifica- 
tions are implemented through the color lookup 
bank 47 as indicated in FIGURE 4. Thus, any 
single original color selected for change to a se- 
75 lected new color, in accordance with a particular 
edit operation, is changed to the new color by 
modifying the R, G, B values in the lookup register 
for the original color and storing the modified val- 
ues in a work lookup register bank. 
20 When the image is thereafter displayed in the 

image window, any screen pixels in the image, or 
within selected limits of the image, that have the 
specified single original color are automatically as- 
signed the modified color in the work register. 
25 Multiple color modifications in a single edit opera- 
tion are simultaneously implemented in the manner 
described for the single color modification. 

An analyzer window control 68 enables the 
operator to employ the computer to provide image 
30 analysis. For example, the operator may request: 
get pixel color or provide histogram. 

Once an edited image becomes acceptable to 
the operator, it is saved by actuating a save selec- 
tor and implementing appropriate save procedures. 
35 The system can then be cleared to its initialized 
state and a new image may be called from storage 
for editing. 

COLOR KEY EDITING CONTROL 

40 

As previously indicated, the pixel selector win- 
dow control 64 (FIGURE 4) includes a color key as 
a means for defining boundaries on the extent to 
which the image stored in the buffer 61 is modified 

45 by a color edit operation. Generally, selective ed- 
iting is achieved by limiting the edit operation to 
selected image pixels. 

Color key editing logically applies colorizer op- 
erations to pixels that obey specific rules about 

so their color. Colors to be edited are designated on 
the screen, and a range of values for each of one 
or more predetermined color descriptor variables is 
determined for the designated colors. In this em- 
bodiment of the invention, the color descriptor vari- 

55 ables are hue, saturation, and brightness. 

Each determined value range operates as a 
pixel rule. That is, image pixels are examined to 
determine whether they fall into or out of each 
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range. Irvrange pixels are modified by a selected 
editing operation. Out-of-range pixels are not modi- 
fied. 

Hue, saturation, and brightness ranges must all 
be satisfied for a pixel to qualify for editing. In 
other applications, however, the rule may be that 
any one or more of the color descriptor variable 
ranges be satisfied for a pixel to qualify for editing. 
Further, descriptor variables other than hue, satura- 
tion, and brightness may be employed. 

Color changes can be made by the color key 
selection procedure or by the conventionally de- 
fined color key editing procedure or by other suit- 
able procedure. In this embodiment, color changes 
are made by the conventionally defined color key 
editing procedure. However, since the invention 
principles can be applied in any of the various 
colorizing procedures, the term "color key editing" 
as used herein is intended to embrace all of the 
applicable colorizing procedures. 

Color editing control means determines RGB 
color values for designated image pixels, converts 
the pixel values to a color descriptor space, prefer- 
ably the hue-saturation-brightness (HSB) space as 
previously indicated, determines HSB value ranges 
for the designated pixels, and modifies only those 
colors in the system color set that fall within the 
HSB ranges. In this embodiment, color modifica- 
tions are made in the color lookup register bank 47, 
also known as a color lookup table (CLUT), only for 
those colors within the HSB ranges. Thus, only 
those image pixels that index the modified color 
registers are modified in color on the screen. 

As previously indicated herein, a full color im- 
age can be encoded for screen display using a 
relatively small color set. 

In this embodiment, a 24-bit image is displayed 
on an 8-bit screen by encoding the full color image 
to a color set consisting of 100 or so colors in 
accordance with the referenced techniques. Each 
pixel of the original 24-bit image is replaced by a 
single 8-bit value (in the encoded image) which is 
an index to an array filled with the RGB values of 
the small color set. The RGB values array is load- 
ed into the CLUT 47 and the encoded image is 
loaded into the image buffer. The monitor display 
hardware uses the CLUT 47 to determine what 
color to display on the screen for each pixel. If the 
RGB values are changed in the CLUT, any pixels 
displayed on the screen that correspond to the 
changed CLUT entries are also changed. 

Only those color entries in the CLUT that have 
RGB values that fall into specific ranges of hue, 
saturation, and brightness are adjusted. The meth- 
od by which these ranges are determined and used 
is illustrated in the flow charts of FIGURES 6A-6L. 

The specific HSB ranges are determined by 
clicking and dragging the previously referenced 
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mouse over areas of the displayed, encoded image 
that have color indicative of the desired ranges. As 
the mouse is dragged around the displayed image, 
the range of hue, saturation, and brightness that is 

5 swept over is tracked, and any requested modifica- 
tions are dynamically applied to the image. Thus, 
range selections are made by sweeping over pixels 
that basically represent the color ranges desired to 
be edited. The application of modifications can 

io occur simultaneously with the selection of ranges. 

A user interface (not shown) to this type of 
editing may comprise sliders for defining the 
amount of hue rotation (in the case of TV color 
editing), brightness change, etc., that is desired. 

75 There is also a set of controls that can be manually 
used to specify the ranges of hue, saturation, and 
brightness. 

As an illustration of color keyed image editing, 
consider the previously referenced image of a tree 

20 with sky showing through the leaves. To adjust the 
color of the sky, the user sweeps the mouse over 
visible sky. As the range of color descriptors is 
defined, each color in the CLUT 47 is checked as 
to whether it lies within the defined ranges. Each 

25 CLUT color is modified only if it lies within the 
defined ranges. The end result is that when the sky 
color selection is completed, the image edit opera- 
tion will have corrected the color of the other 
visible sky as well as tiny bits of sky showing 

30 through the leaves. 

The mechanism by which the user selects the 
range of colors to be adjusted is an important 
aspect of the present invention. Thus, the user 
click-and-drags the mouse in a region of the dis- 

35 played image that represents the desired colors. 
As the mouse is dragged, display pixels in a small 
area (e.g. 5x5 block) under the mouse are analyzed 
and stored as a brightness, a hue, and a saturation. 
As each new color is stored, ranges of hue, 

40 saturation, and brightness are updated. The ranges 
are the minimum ranges which encompass all of 
the stored colors. All colors in the CLUT 47 are 
tested to determine whether they lie within these 
ranges, and, if so, the entries are adjusted accord- 

45 ing to the selection algorithm. 

Once the color ranges are defined interactively, 
they can be used to apply a color-key edit on a full 
color, non-encoded image. Results appear best for 
images such as line art that contain large areas of 

50 constant color. 

To improve the results of applying interactively 
defined color ranges to full color, non-encoded 
images, the selection algorithm is modified as fol- 
lows. While tracking the colors in the small block 

55 under the mouse, the average color of the block 
and the standard deviation of the colors are also 
tracked. When applying color-key editing to a full 
color image, use ranges specified by the color 

7 



13 EP 0 536 892 A1 14 



value + K times the standard deviation are used. 
The value of K may be set by image heuristics. 

Each pixel in the display image is represented 
by an integer index to the CLUT 47. The actual 
display color (in RGB values) is found by reading 
the RGB values from the CLUT 47. If a tone 
reproduction correction has also been applied to 
linearize the display, the correction curves are in- 
verted to extract actual colors from the corrected 
colors. 

COLOR KEY EDITING PROCEDURE 

The preferred procedure for color key editing is 
illustrated by an overview flow chart 100 in FIGURE 
6A. After a color key selector is operated, the color 
key procedure 100 is entered, and once the mouse 
is detected to be down by test block 102, the 
mouse position is determined by block 104. 

Pixel values for a small pixel block about each 
successive mouse position are determined from 
the image buffer 61 by a block 106. The pixel 
block contains 25 pixels in the preferred embodi- 
ment. Next, each pixel value is converted to hue, 
saturation, brightness (HSB) values in a block 108. 
The H wraparound, S, and B ranges are tracked by 
respective blocks 110, 112,and 114. 

A functional block 116 then modifies those 
CLUT registers having a color within the deter- 
mined ranges. The color modifications are made in 
accordance with color value specifications provided 
by whatever edit operation is being used by the 
user. 

In test block 117, a determination is made as 
to whether a statistical option has been exercised 
by the user for full color editing. If not, a return is 
made to the mouse test block 102 to recycle 
through the procedure 100 to process the next 
input pixels or to end in block 125 if the mouse has 
been removed. 

If the statistical option has been exercised at 
block 117, data values are collected for analysis by 
statistical means. The analysis may involve use of 
standard deviations, histograms or other statistical 
tools and it may employ an optimizing procedure 
depending on the needs of the particular applica- 
tion. In this embodiment, the full color edit option 
employs an averaging procedure. 

Accordingly, the average HSB values are cal- 
culated by block 118. Blocks 120, 122, and 124 
track average H wraparound range, average S 
range, and average B range, respectively, for the 
reasons previously presented. After option process- 
ing, a return is made to the mouse test blocks to 
process the next mouse position by the procedure 
100. 

The hue range is tracked with an algorithm 
different from the algorithm employed for tracking 



saturation range and brightness range. The basic 
reason is that hue is normally defined in a circular 
domain, namely an angle that has no absolute 
minimum or maximum. For example, counterclock- 
5 wise angular rotation beyond - 180 degrees wraps 
around to + 179 degrees. 

A preferred algorithm employed in the block 
110 for tracking the range of a circular domain 
variable is illustrated in FIGURE 6E. A preferred 
70 algorithm for tracking the linear domain variables S 
and B is illustrated in FIGURE 6G. If desired, linear 
domain variables can be treated as a circular do- 
main and tracked accordingly. 

In FIGURE 6B, the pixel value conversion pro- 
75 cedure is shown in greater detail. Thus, block 126 
uses the pixel value index to read RGB values from 
the CLUT 47. Test block 128 determines whether 
the displayed image is linearized, and, if so, block 
130 uses inverted linearization tables to convert the 
20 correctively linearized RGB values to actual RGB 
color values. 

Block 132 employs the listed equations to cal- 
culate the HSB values from the actual RGB values. 
HSB values can be based on any of various 
25 luminance-chrominance color spaces such as 
CIELAB, CIELUV, YIQ, etc. In this embodiment, 
Xerox/yes Linear color space is employed. 

As indicated previously, a special procedure is 
needed for dynamic tracking of a circular variable, 
30 and in particular for tracking hue in this embodi- 
ment. A circular variable has no minimum or maxi- 
mum value because it "wraps around" Consider a 
hue range 140 shown in FIGURE 7B1. A linear 
representation 142 of the range 140 is shown in 
35 FIGURE 7B2 with hue measured from - 180 de- 
grees to + 1 80 degrees. 

The range appears to be split due to the 
wraparound. This would conventionally double the 
conditional checking to determine acceptable val- 
40 ues in the desired range. This particular 
wraparound can be avoided by measuring hue 
from 0 to 360 degrees as represented at 144 tn 
FIGURE 7B3, but a problem still remains when 
going through 0 degrees. Thus, simple tracking of 
45 the minimum and maximum hue value is inad- 
equate for the measurement of a hue range. 

The procedure employed expects a sequence 
of hue values as input. The hue values need not be 
ordered in any particular fashion. The actual hue 
5a value can be mapped into an integer value for 
speed such as integers - 180 to 179 or 0 to 255 for 
single byte storage. 

As each hue value is captured, it is inserted 
into a sorted linked list 146 of data structures 
55 (FIGURE 7C). The list 146 contains exemplary data 
values, and it always has less than or an equal 
number of elements in the components range. 
Each data structure contains the hue value, the 
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number of times this particular hue value has oc- 
curred, and the distance to the next entry in the 
list. The distance to the next entry is the difference 
between the hue value of the next list entry and the 
current hue value. 

The distance for the last item in the list is 
calculated at the end of inputting values. The dis- 
tance assigned to the last entry in the list is the 
sum of the distance to the maximum hue value and 
the distance from the minimum hue value to the 
first list entry. 

To determine the selected range, the list is 
searched for the entry A with the largest distance 
value, indicating the longest range in which no hue 
values have occurred. The next entry is B. If A is 
the last entry in the list, B is the first entry in the 
list. 

The selected hue range is the complement of 
the range Hue A to Hue B , i.e., the selected range is 
the range Huee to Hue A - If Hues is greater than 
Hue A , then the range wraps around. Another way to 
determine wraparound is: if Hue A is not the last 
structure in the list, wraparound exists. 

A particular hue is tested to determine whether 
it is in the selected range as shown by a procedure 
in block 148 of FIGURE 7A. In the procedure 
shown in block 148, the only difference between 
the wraparound case and the non-wraparound case 
is the use of a logical OR instead of a logical AND. 
In the example data in the list 146, the longest 
distance corresponds to the middle entry B. There- 
fore the selected range is from the rightmost entry 
C to A (the leftmost entry), i.e., from a hue of 65 to 
a hue of - 150. Since 65 is greater than - 150, the 
selected range must wraparound. 

FIGURE 6E shows the preferred wraparound 
track procedure 110 in greater detail. Generally, 
each hue value is stored in a sorted list of records 
150 (FIGURE 6C) and the largest domain gap 152 
(FIGURE 6D) that contains no values is identified. 
The tracked range is the region of the circular 
domain outside the largest gap. 

With reference to FIGURE 6C, each unique 

incoming hue value is stored in the "entry list". 

Each member of the list contains several items of 
information. Entries inserted in the list are sorted in 
ascending order. Therefore, the value associated 
with "entry" is a larger number than the value 

associated with "prev entry n t and is less than the 

value associated with "next entry". If an entry 

already exists with a value equal to the incoming 
value, then the "count" field of the entry is incre- 
mented by one. This allows the total number of a 
specified value to be tracked for proper calculation 
of averages. With reference to FIGURE 6D, 

"max value" is the maximum value the incoming 

value can take before wrapping to the 
"min value"; for example, the max value is 179 
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degree and the min value is -180 degrees for 

tracking of ranges in hue in the present embodi- 
ment. 

With reference to FIGURE 6E, the wraparound 
5 track procedure 1 1 0 is entered with a test made in 
block 160 to determine whether the entry value is 
already in the entry list. If so, block 164 advances 
the entry count for that value by one. The process- 
ing then breaks in block 166. 
io Any entry not found in the list is processed by 

block 168 where a new entry is made and the entry 
count is set to one. In block 170, the new entry is 
inserted in the list with sorting by value in ascend- 
ing order. 

75 If test block 172 determines that the current 

entry is the last entry, block 174 calculates the 
entry distance as the maximum value less the 
entry value plus the first entry value less the mini- 
mum value. If the current entry is not the last entry, 

20 block 176 makes the entry distance equal to the 
next entry value less the current value. 

Test block 178 next determines whether the 
current entry is the first list entry. If so, block 180 
makes the last entry distance equal to the maxi- 

25 mum value less the last entry value plus the entry 
value less the minimum value. If not, block 182 
makes the previous entry distance equal to the 
entry value less the previous entry value. After 
execution of the block 166 or 180 or 182, program 

30 execution returns to the main procedure 100. 

A procedure 190 shown in FIGURE 6F is em- 
ployed to determine whether a hue value is in the 
wraparound range. Generally, to determine whether 
a given value is within the selected range of a 

35 circular domain, the entry list 150 structure 

(FIGURE 6C) that has been built is analyzed. 

First, the entry list 150 is scanned to deter- 
mine which entry has the largest distance field and 
denote this entry as A. The distance field indicates 

40 how big a gap (in units of "value") exists to the 

next entry in the entry list 150. 

The next entry in the entry list 150 is des- 
ignated as entry B. The selected range is then 
defined as all values that do not exist in the gap 

45 between values A.value and B. value. 

Block 196 (FIGURE 6F) determines the maxi- 
mum entry distance. In block 198, the maximum 
distance entry is made equal to A. 

Block 200 tests whether A is the last entry. If 

50 so, block 202 makes B equal to the first entry. If 
not, block 204 makes B equal to A.NEXT. 

Test blocks 206, 208, and 210 check the rela- 
tionship between the current value and A.value and 
B.value to determine whether the current value is in 

55 range. Thus, if B.value is less than A.value and 
block 210 indicates B.value is less than value and 
value is less than A.value, the value is recorded in 
range in block 214. However, if current value is 

9 
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less than B.value or greater than A.value, block 21 6 
records the current value as out of range. 

If A.value is less than B.value (block 208), and 
if B.value is less than or equal to current value or 
current value is less than A.value, the current value 
is recorded in range for the block 214. If not, the 
block 216 records the current value as out of 
range. If the block 208 tests false, an error is 
generated. Block 218 ends the range search proce- 
dure after each entry is processed. 

A track range procedure 113 used in the 
blocks 112 and 114 (FIGURE 6A) to track B and S 
values is shown in FIGURE 6G. The linear bright- 
ness and saturation domains are tracked using 
minimum and maximum values. Thus, test block 
230 checks whether the current value is less than 
the minimum value. If so, the value is set equal to 
the minimum value in block 232. 

If not, the current value is tested against the 
maximum value by block 234. If the current value 
is greater than the maximum value, block 236 
makes the value equal to the maximum value. 
Otherwise, the current value is between the mini- 
mum and maximum values and a return is made to 
the main procedure 1 00. 

A value-in-range procedure 250 (FIGURE 6H) 
is used in the present embodiment to determine 
whether domain values are within the selected 
range by comparison with current minimum and 
maximum values. Thus, test blocks 252 and 254 
respectively check whether the value is greater 
than or equal to the minimum value and less than 
or equal to the maximum value. 

If so, block 256 records the value in range. If 
either test is not true, block 258 records the value 
as not in range. 

The procedure 250 is then ended by block 260. 

If desired, the encoded image editing is suit- 
ably extended to the full color image. In this man- 
ner, the full color edited image is made available 
for printing or other uses that require high resolu- 
tion images. 

Thus, the color key editing control includes a 
selectable option in which additional statistical data 
is tracked to improve the effectiveness of the color 
key editing algorithm in the HSB procedure 118 
(FIGURE 6A) when it is applied to continuous-tone 
24- bit images as opposed to encoded images. 
The color key editing control can be requested to 
implement the option, and menus are provided for 
tracking HSB data for any of various statistical 
analysis procedures. 

The average value of HSB is tracked for each 
small pixel block that is analyzed. As a result, in 
addition to tracking the global minimum and maxi- 
mum HSB for all pixels within all small blocks, 
minimum and maximum average HSB values are 
tracked. The latter values better represent the color 



ranges that exist in the 24-bit image, and are used 
to define the ranges when an edit application is 
requested on a 24-bit image. 

The average HSB procedure is shown in great- 
5 er detail in FIGURE 6J. For each mouse position, 
neighboring pixels in a small box around the posi- 
tion are processed to determine HSB values. Ac- 
cordingly, block 270 gets the pixel color values for 
all 25 pixels in the small block. 
ro Pixel color values are then converted to HSB 

values by block 272. In block 274, H wraparound 
range is tracked and B and S are calculated for the 
current pixel block as shown. Average B and S per 
block pixel are calculated in block 276 by dividing 
75 the pixel block totals by the number of pixels in the 
block. Average H wraparound is obtained from an- 
other procedure 280 shown in FIGURE 6K. 

In the average hue procedure 280, block 288 
determines the maximum distance entry. Block 289 
20 then makes A equal to the maximum distance 
entry. A test is made in block 290 to determine 
whether A equals the last entry. If so, B is made 
equal to the first entry in block 292 and block 294 
tests whether B is less than A. If not, block 291 
25 sets B equal to A.NEXT. 

If B is less than A, block 296 records a false 
wraparound and sets total count equal to zero and 
sum equal to zero. Next, block 298 determines the 
sum and total count for all entries in the entry list 
30 using the equations shown. 

If B is not less than A, block 300 records a true 
wraparound with sum and total count both equal to 
zero. Next, for all entries in the entry list starting at 
the first entry and ending with A, block 302 uses 
35 the indicated equations to calculate sum and total 
count. Block 304 the uses the indicated equations 
to calculate sum and total count for all entries in 
the entry list starting with B and ending with the 
last entry. 

40 In block 306, the average hue value is cal- 

culated by dividing the sum by the total count for 
each entry from the blocks 298 and 304 for false 
and true wraparounds. 

If the average hue value is less than the mini- 

45 mum value as checked in block 308, the average 
hue value is set equal to the average value plus the 
maximum value less the minimum value by block 
310. The procedure 280 ends in block 312. 

Once ranges in HSB have been selected, ed- 

50 iting changes can be applied by modifying the 
color lookup tables 47. As previously indicated, the 
color lookup tables or CLUT are an array of RGB 
values. If the RGB value stored into the k-th entry 
of the CLUT array is changed, any displayed pixels 

55 that have a pixel value of k will be changed. One 
example of such editing is applying a hue rotation 
much like adjusting the hue on a television set. A 
hue rotation can be considered to be the addition 
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of some A WB to brightness and ^WS to satura- 
tion. 

The CLUT modification procedure is shown in 
greater detail in FIGURE 6L. First, block 320 con- 
verts RGB values to HSB values for all CLUT 
registers. Test blocks 322,324, and 326 then deter- 
mine whether the values for the currently pro- 
cessed CLUT register is within the respective H, S, 
and B ranges. 

In this embodiment, if all three tests are true, 
the new value of H or S or B is set equal to the old 
value plus the editing change in block 327. If any 
one test is false, the new value of H or S or B is 
set equal to the old value and block 325 recycles 
to the block 320 for the next register modified in 
the working registers in the CLUT or ends in block 
329 if the last register in the modified or working 
register bank has been processed. 

Block 328 converts the new HSB values to 
RGB values for each CLUT element, and the new 
RGB values are restored to the CLUT 47 (FIGURE 
4), and the employed edit operation applies the 
resulting color set in the CLUT to the image pixels. 
Color changes occur for those pixets that index 
CLUT registers for which color changes have been 
made as a result of the range checking steps in the 
blocks 322, 324, and 326. After HSB conversion, a 
return is made to the main procedure 100. 

In FIGURES 8A-8D, there are shown displays 
for four different exemplary cases of color key 
editing. In each case, an image window is shown 
with its top level menu and an image as edited by 
the particular editing control applied by an operator 
through an editor control window. 

In FIGURES 8A-8C, the selected edit operation 
from the menu is TV Image, and the active areas 
employ the color dot graphic and the brightness- 
contrast checkerboard graphic with their crossed 
hairline overlays. The values for relative hue and 
saturation and brightness and contrast are dis- 
played below the graphic areas. 

FIGURE 8D is similar but it illustrates a paint 
selection for the edit operation. The graphic is the 
color wheel with rotating radial line overlay and 
adjustable concentric ring overlay. The selected 
paint color is displayed in an active area box be- 
fore it is applied to the image. Values of H, S and 
V (B) for the selected color are displayed. In this 
case some brushing of the selected yellow color 
has been done on the girl's cheek in the image. 

Claims 

1. An image editor for an image processing sys- 
tem having a predefined set of system colors, 
said editor comprising: 

means for storing color pixel data for an 
image to be edited; 



means for displaying the stored image on 
a monitor screen; 

means for designating displayed colors 
that a user desires to modify in a region of the 
5 displayed image; 

means for detecting pixel position data 
corresponding to the designated colors; 

means for determining and storing values 
for at least one color variable from said color 
io pixel data for said pixel position data; 

means for determining a minimum range 
of values for said color variable that encom- 
pass colors designated by said designating 
means; and 

15 means for applying a preselected color 

edit operation to those image pixels having 
colors corresponding to those color variable 
values within the minimum range. 

20 2. An image editor as claimed in claim 1, 
wherein: 

said determining and storing means deter- 
mines and stores respective values for multiple 
color descriptor variables; 
25 said range determining means determines 

a minimum range of values for each of the 
color descriptor variables; and 

said applying means applies the edit op- 
eration to image pixels having colors corre- 
30 sponding to those color descriptor variable val- 

ues within at least one of the minimum ranges. 

3. An image editor as claimed in claim 1 or claim 
2, wherein: 

35 said designating means includes means 

for specifying successive image positions hav- 
ing the desired color(s); and 

said position detecting means detects pix- 
els in a small image area around each speci- 

40 fied position. 

4. An image editor as claimed in any one of the 
preceding claims, wherein: 

the color descriptor variables are hue, 
45 saturation and brightness; said color pixel data 

storing means includes an image pixel buffer 
and a lookup color register bank; 

said lookup color register bank having a 
modifiable register for storing basic color com- 
so ponent values for each color in the set of 

system colors; 

said image pixel buffer having image pixel 
registers each of which is indexed to a particu- 
lar lookup color register to display the color 
55 required for that pixel; 

said edit applying means includes means 
for testing each of said modifiable lookup-table 
registers to determine those stored color val- 
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ues that correspond to hue, saturation and 
brightness values within the minimum ranges 
therefor; and 

said edit applying means further includes 
means for modifying those modifiable lookup- 5 
table registers having in-range hue, saturation 
and brightness values in accordance with the 
edit operation. 

5. An image editor as claimed in claim 3, 10 
wherein: 

the color descriptor variables are hue, 
saturation and brightness; 

means are provided for determining re- 
spective average values of hue, saturation and 75 
brightness (HSB) for pixels in each of said 
small image areas; and 

said range determining means determines 
a minimum range of average values for each of 
said HSB descriptor variables for application to 20 
full color edit operations. 

6. An image editor as claimed in claim 3, 
wherein: 

means are provided for statistically analyz- 25 
ing color descriptor variable values to provide 
respective statistical values for each descriptor 
variable for pixels in each of said small image 
areas; and 

said range determining means determines 30 
a minimum range of statistical values for each 
of said color descriptor variables for application 
to full color edit operations. 



7. An image editor as claimed in claim 6, wherein 
said statistical analyzing means provides aver- 
age values for each descriptor variable. 



35 



8. An image editor as claimed in claim 1, 

wherein: ^ 

said one color variable is a color descriptor 
variable having values defined in a circular 
domain; and 

said range determining means includes: 

means for sorting said stored values for . 45 
said variable in value order and for determining 
the greatest value distance between adjacent 
stored values A and B; 

means for comparing the A and B values 
to determine whether the selected range is or 50 
is not a wraparound range; and 

said applying means includes means for 
determining whether each pixel has a color 
corresponding to a color descriptor variable in 
the selected wraparound or non-wraparound 55 
range. 



9. A range tracker for a variable having values in 
a circular domain, said tracker comprising: 

means for sorting stored values of said 
variable in value order and for determining the 
greatest value distance between adjacent 
stored values A and B; 

means for comparing the A and B values 
to determine whether the selected range is or 
is not a wraparound range; and 

means for determining whether each value 
in another set of values for said variable is in 
the selected wraparound or non- wraparound 
range. 

10. An image editor as claimed in claim 8 or a 
range tracker as claimed in claim 9, wherein: 

said determining means determines a val- 
ue to be in range if it is greater than the B 
value and less than the A value if the selected 
range is a non-wraparound range; and 

said determining means determines a val- 
ue to be in range if it is greater than the B 
value or less than the A value if the selected 
range is a wraparound range. 

11. A method for editing an image in an image 
processing system having a predefined set of 
system colors, the steps of said method com- 
prising: 

storing color pixel data for an image to be 
edited; 

displaying the stored image on a monitor 
screen; 

designating displayed colors that a user 
desires to modify in a region of the displayed 
image; 

detecting pixel position data corresponding 
to the designated colors; 

determining and storing values for at least 
one color variable from said color pixel data for 
said pixel position data; 

determining a minimum range of values for 
said color variable that encompass colors des- 
ignated by said designating step; and 

applying a preselected color edit operation 
to those color variable values within the mini- 
mum range. 

12. A method as claimed in claim 11, wherein: 

said determining and storing step deter- 
mines and stores respective values for multiple 
color descriptor variables; 

said range determining step determines a 
minimum range of values for each of the color 
descriptor variables; and 

said applying step applies the edit opera- 
tion to image pixels having colors correspond- 
ing to those color descriptor variable values 
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